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METHOD AND APPARATUS FOR DELIVERING AN ELECTRONIC MAIL 



MESSAGE WITH AN INDICATION OF THE PRESENCE OF THE SENDER 



5 Cross-Reference to Related Applications 

The present application is related to United States Patent Application entitled 
"Method and Apparatus for Delivering a Voice Mail Message With an Indication of a Presence 
Status of a Sender," (Attorney Docket Number 502054) and United States Patent Application 
entitled "Programmable Presence Proxy for Determining a Presence Status of a User," (Attorney 
10 Docket Number 502084), each incorporated by reference herein. 

Field of the Invention 

The present invention relates generally to methods and systems for delivering 
electronic mail messages over a network, and more particularly, to methods and systems that 
15 deliver electronic mail messages to one or more intended recipients with an indication of the 
presence or availability of the sender. 

Background of the Invention 

Electronic mail (email) and instant messages (IMs) have become popular means 
20 for communicating. An email or instant message generally comprises a message body and one 
or more indicated recipients. While an instant message is generally routed in real-time to the 
indicated recipients, email applications, such as Microsoft Outlook, are typically not enabled to 
provide responses in real time. Even with an instant message, routing is "instant 1 " only within 
the parameters of the network(s) used to deliver the message and is subject to network delays, as 
25 well as reliability characteristics of the network. 

Currently available email applications typically only allow the email recipient to 
respond to the sender using a reply email, which is not a real time method of communication. If 
the email recipient would like to respond in real-time, the recipient generally must manually 
look-up the desired contact information of the sender and decide how to best reach the sender at 
30 that time (for example, by calling the sender at home or in the office). Even after deciding how 
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to best reach the intended person, the real-time communication attempt can still fail, for example, 
if the intended person is not there or is unavailable. 

Instant messaging systems generally provide a mechanism for determining 
whether a message recipient is "present." See, for example, Atkins et al., "Introducing Instant 

5 Messaging and Presence Into the Workplace," Proc. of the Conf. on Human Factors in 
Computing Systems, Minneapolis, Minnesota, USA, ACM CHI 2002 (April 20, 2002), 
downloadable from http://www.informatik.uni-trier.de/-ley/db/conf7chi/chi2002.html. The 
instant messaging system provided by America Online, for example, provides both an instant 
message function and a presence awareness function. The presence information allows the 

10 recipient of an instant message to determine whether the sender of the instant message is 
currently available (i.e., logged on to the AOL service) to receive additional instant messages. In 
addition, a number of instant messaging systems allow a user to provide a text message 
indicating his or her current availability, such as "out to lunch," or "in a meeting." Thus, the 
users of instant messaging systems can make more informed decisions about how to best 

15 communicate with an intended recipient. 

Unlike instant messaging systems, however, email systems typically do not allow 
an email recipient to determine whether the message sender is currently present. In addition, 
neither instant messaging systems nor email systems permit a message recipient to automatically 
respond in real-time to the message sender using a non-textual form of communication, such as 

20 automatically placing a telephone call to a telephone terminal at which the intended person is 
believed to be present. A need therefore exists for methods and systems that deliver electronic 
mail messages to one or more intended recipients with an indication of the presence of the 
sender. A further need exists for electronic mail methods and systems that allow a message 
recipient to automatically respond to a message sender using a non-textual form of 

25 communication. 

Summary of the Invention 

Generally, a method and apparatus are disclosed for delivering electronic mail 
messages to one or more intended recipients with an indication of the presence of the sender. 
30 The provided presence information allows the recipient to better determine the best way to 
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respond to the communication. The presence information may indicate, for example, if the 
message sender can currently be reached by email, instant message, a telephone call or voice 
mail message. The present invention increases productivity by enabling a better selection of the 
best way to respond to an email communication. If the message sender is present for a real time 

5 communication, for example, the message recipient can choose a real time or near real time 
mode of communicating, such as a telephone call or instant message. Otherwise, the message 
recipient can select a non-real time mode of communicating, such as a reply email message, 
voice mail message or a page. 

According to another aspect of the invention, a message recipient can 

10 automatically respond to a text-based email message using a non-textual form of communication, 
such as a telephone call to a telephone where the sender is believed to be present. Furthermore, 
the presence server can provide presence information across domains so that a user in one 
domain (such as the customers of one Internet Service Provider) can automatically respond in 
real-time or non-real time to a user in another domain (such as the customers of another Internet 

15 Service Provider). In one implementation, a client-side presence enabled email application 
process (i) retrieves email messages from an email server of a user, (ii) queries a presence server 
to determine the presence of the sender of each retrieved email message, and (iii) presents the 
retrieved email message(s) to the recipient, for example, in an u in-box," with an indication of the 
presence of the sender. In addition, the disclosed presence enabled email application process 

20 allows the message recipient to automatically respond, for example, by an email, instant 
message, telephone call or voice mail message to a device where the sender is believed to be 
present. 

A more complete understanding of the present invention, as well as further 
features and advantages of the present invention, will be obtained by reference to the following 
25 detailed description and drawings. 

Brief Description of the Drawings 

FIG. 1 illustrates a network environment in which the present invention can 

operate; 
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FIG. 2 is a schematic block diagram of an exemplary recipient device of FIG. 1 
incorporating features of the present invention; 

FIG. 3 is a schematic block diagram of an exemplary presence server of FIG. 1 
incorporating features of the present invention; 
5 FIG. 4 is a sample table from an exemplary presence database of FIG. 1 ; 

FIG. 5 illustrates an exemplary "login" process that can be used by users to 
register with the presence server of FIG. 3; 

FIG. 6 is a flow chart describing an exemplary implementation of the client-side 
presence enabled email application process of FIG. 2; and 
10 FIG. 7 is a user interface incorporating features of the present invention. 



Detailed Description 

FIG. 1 illustrates the network environment in which the present invention can 
operate. As shown in FIG. 1, a sender employing a sender device 110 sends an email message 

15 over a network 120 to one or more intended recipients, each employing a corresponding 
recipient device 200-1 through 200-N, discussed below in conjunction with FIG. 2. The email 
message is typically delivered to a "mailbox" of an email server 130 associated with a 
corresponding recipient. The recipient must generally log into the email server 130 to access the 
email messages, in a known manner. The network(s) 120 may be any combination of wired or 

20 wireless networks, such as the Internet and the Public Switched Telephone Network (PSTN). 
The email server 130 typically serves a community of users, such as the employees within an 
enterprise or the customers of an Internet Service Provider. While the present invention is 
described in the context of an email message system, it will be understood by those of ordinary 
skill in the art that the present invention encompasses other types of messages and is not limited 

25 to email messages. 

According to one aspect of the invention, an email message is delivered to each 
recipient with an indication of the presence or availability of the sender. As used herein, the 
term "presence" shall mean the representation of a state characterizing the existence of an active 
device through which a user can communicate or through which presence can be detected. The 

30 state is specific to a particular communication service, such as email , voicemail or instant 
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messaging, or presence detection service, such as audio-visual detection, GPS devices or heat 
sensors. Users may have multiple active communication devices. 

The provided presence information allows the recipient to better determine the 
best mode of communication to use when responding to the communication. In this manner, 

5 productivity is enhanced by enabling a better selection of the best way to respond to the email 
communication. If the message sender is present for a real time communication, for example, 
the message recipient can choose a real time or near real time mode of communicating, such as a 
telephone call or instant message. Otherwise, the message recipient can select a non-real time 
mode of communicating, such as a reply email message, voice mail message or a page. This 

10 informed choice leads to a more efficient, productive and cost effective communication. 
According to another aspect of the invention, a message recipient can automatically respond to a 
text-based email message using a non-textual form of communication, such as a telephone call to 
a telephone where the sender is believed to be present. 

A sender that wishes to send an email message employs a text-enabled sender 

15 device 1 10, such as a personal computer or personal digital assistant, to enter the email message. 
As previously indicated, an email message generally comprises a message body and one or more 
indicated recipients. The email message is received by the email server 130 over the network 
120 and is routed to the mail box corresponding to each indicated recipient. In one exemplary 
implementation, a client-side presence enabled email application process 600, as discussed 

20 further below in conjunction with FIG. 6, associated with each recipient (i) retrieves email 
messages from the email server 130, (ii) queries a presence server 300, discussed below in 
conjunction with FIG. 3, to determine the presence of the sender of the retrieved email message, 
and (iii) presents the retrieved email message to the recipient, for example, in an "in-box," with 
an indication of the presence of the sender. 

25 In addition, the presence enabled email application process 600 allows the 

message recipient to automatically respond, for example, by an email, instant message or 
telephone call to a device where the sender is believed to be present. In the context of an email 
system, presence information is generally most useful if presented to the email recipient when 
the recipient is reading the email. However, other variations are possible and within the scope of 

30 the present invention. Generally, the presence server 300 records presence information for each 



-5- 



502063-A-01-US (Khakoo) 

user in a given community, such as the availability of each user to receive email messages, 
instant messages or telephone calls to one or more indicated telephone numbers. 

It is noted that the community served by the email server 130 is typically not the 
same community as that served by the presence server 300. In fact, it is an object of the present 

5 invention that the presence server 300 is based on an open standard that provides presence 
information beyond any single community served by a single email server 130. Thus, the 
presence server 300 can provide presence information across domains so that a user in one 
domain (such as the customers of one Internet Service Provider) can automatically respond in 
real-time or non-real time to a user in another domain (such as the customers of another Internet 

10 Service Provider). In this manner, the presence server 300 is not tied to any given email system, 
and thus the presence server 300 can support any email client. In an alternate server-side 
implementation, the email server 130 can automatically provide the email message with an 
indication of the presence of the sender. 

FIG. 2 is a schematic block diagram of an exemplary recipient device 200 

15 incorporating features of the present invention. The recipient device 200 may be any text- 
enabled computing device, such as a personal computer or personal digital assistant. As shown 
in FIG. 2, the exemplary recipient device 200 includes a processor 215 and a memory 202, in 
addition to other conventional elements (not shown). The processor 215 operates in conjunction 
with the memory 202 to execute one or more software programs. Such programs may be stored 

20 in memory 202 or another storage device accessible to the recipient device 200 and executed by 
the processor 215 in a conventional manner. For example, as discussed below in conjunction 
with FIG. 6, the memory 202 may store a presence enabled email application process 600 that (i) 
retrieves an email message for the recipient associated with the device 200 from the email server 
130, (ii) queries a presence server 300, discussed below in conjunction with FIG. 3, to determine 

25 the presence of the sender of the retrieved email message, and (iii) presents the email message to 
the recipient with an indication of the presence of the sender. A suitable user interface that may 
be employed by the presence enabled email application process 600 to present retrieved email 
messages, together with an indication of the presence of the sender, is discussed below in 
conjunction with FIG. 7. 
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As previously indicated, the presence server 300 records presence information for 
each user in a given community. For example, the recorded presence information may include 
the availability of each user to receive email messages, instant messages or telephone calls to one 
or more indicated addresses or telephone numbers. The presence server 300 can track real time 

5 changes in the presence status of each user that is used by the present invention to provide better 
communication when responding to an email message. The exemplary presence server 300 is 
implemented in accordance with the specifications of the emerging PAM architecture, described, 
for example, at www.pamforum.org. For example, the presence server 300 may be embodied in 
accordance with the teachings of David Boyer et al., "Presence Awareness for Future 

10 Telecommunication Systems", Chapter in Virtual Reality Technologies for Future 
Telecommunications Systems, Algirdas Pakstas and Ryoichi Komiya (Eds.), John Wiley & Sons, 
LTD, (2002); or Mark Handel et al., "Requirements for Presence Awareness: The RVM Model," 
http://www-personal.si.umich.edu/~ha (or 
accessible through a keyword search on google.com), each incorporated by reference herein. 

15 As shown in FIG. 3, the presence server 300 includes a client connection module 

320 that is responsible for managing client connections. The client connection module 320 
facilitates communication between the presence server 300 and each client. In the exemplary 
implementation shown in FIG. 3, the client connection module 320 supports three client 
interfaces 310-1 through 310-3. A first client interface 310-1 provides a 'persistent' interface 

20 for presence applications. A persistent connection is maintained between the client and the server 
300. A heartbeat mechanism can be utilized to make the system robust to network outages. 
Notifications are also sent via the first client interface 310-1. If a subscribed-to-users presence 
status changes (a new device is now available for communication), the user's client is sent a 
notification to indicate this. 

25 A second client interface 310-2 supports non-persistent User Datagram Protocol 

(UDP) communications via a Session Internet Protocol (SIP) proxy that provides notifications 
and registration to a well-known port number via the SIP notification and register protocol. See, 
J. Rosenberg et al., "SIP Extensions for Presence," IETF Internet Draft, dragft-rosenberg-impp- 
presence-00.txt (June 15, 2000). A third non-persistent client interface 310-3 supports "thin" 

30 Web clients. A thin client does not support notifications. The client queries the server 300 
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periodically to see if the presence status of any of the parties that the user subscribes to has 
changed. 

In addition, the presence server 300 includes a subscription management module 
325 that is responsible for managing subscriptions. A subscription list, often referred to as a 

5 "buddy list," is a list of the people, groups and Web pages to whose presence and availability a 
user has subscribed. Examples include a stock price when it hits a certain level, the availability 
of a document when it is ready and the nearest fax machine that is not in use. A user might also 
subscribe to different applications that a user has access to or features of systems that change. 
For example, a user may want to know when someone hangs up his or her telephone (the identity 

10 subscribes to an agent's on-hook field). Subscriptions should also be permitted to an agent's 
presence information that might be considered to be networked appliances. For example, a 
homeowner could subscribe to a remote electronic doorbell. 

The subscription management module 325 has a number of related modules, that 
let a user manage groups and buddy lists. The subscription management module 325 supports the 

15 availability of specific communication capabilities and, given the right permissions, a user can 
receive presence information about specific communications capabilities. A presence 
management module 335 allows a presence client to register or unregister its presence. Different 
clients can register unique devices and capabilities for a given user. Some clients can detect 
when a user has been idle. The presence module 335 is updated when an idle threshold is 

20 reached. 

A presence notification module 345 notifies the clients about the presence change 
of other clients (or devices), that subscribed to the presence of the client. Notifications of 
presence state changes are sent to subscribed and on-line watchers (for the interfaces that support 
notifications). Users are also notified when someone they have been watching changes their 

25 accessibility to that user. If a user stops allowing a watcher to receive his/her presence 
information, the watcher is notified of this change in real time. This also applies to groups. The 
watcher of a group is sent a notice when his/her group membership is terminated. 

An active object management module 340 maintains a list of currently connected 
clients and synchronizes the information with the data store. The object management module 340 

30 also tracks active groups. When a user logs on, all groups that the user is a member of are 
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updated to indicate his/her presence. If the user is the first present member in the group, the 
group now becomes active. 

An access control list (ACL) management module 330 allows the users to manage 
their access control lists. An access control list lets a user indicate who they will allow to 

5 "watch" them (i.e., receive his or her presence information). It is important in an enterprise 
setting, for example, to allow users to specify who (other users and groups) is permitted to 
receive their presence data (note that this does not mean that all the users on the list will actually 
elect to subscribe to this user). Both 'Allow Lists' (no one except X, Y and Z is allowed access 
to my presence and availability information) and 'Deny Lists' (everyone except X, Y and Z is 

10 allowed access to my presence and availability information) are typically needed for Enterprise 
applications. Alternatively, a system might require users to grant a user's request for presence 
data in real time — a user is sent a message saying someone wants to add them to their 
subscription list and is asked to grant or not grant permission. 

In most systems, a user receives a notification when a new user wants to receive 

15 their presence information. This requires an explicit action each time a user wants to reject the 
subscription of another user to their information. In an enterprise setting, this may not be 
appropriate. An ACL system is used that allows only those users and groups to receive 
information for which this permission had been initially granted. Users can, if they desire, 
toggle this setting so that everyone gets their presence information except those who are 

20 explicitly listed as people who should not be permitted access to such data, in a known manner. 
For a group, the ACL list is used to indicate who is allowed to join the group. The member list 
is a list of those users who have actually joined the group. A group may be are open for anyone 
to join or may have a list of people who are allowed to join; yet everyone on the list may not 
elect to join the group. Groups can also have a separate subscription list. 

25 A datastore accessing module 360 provides a common interface through which all 

the data access takes place. A Lightweight Directory Access Protocol (LDAP) datastore 380 is 
actively supported. The LDAP datastore 380 is a persistent repository for storing the 
information about the objects registered with the server. It is noted that new fields can be added 
to any object by a client. New fields do not require any changes to the presence server 300 (new 

30 fields are automatically created). These fields can also be subscribed to by a client. 
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The LDAP datastore 380 provides a presence database 400, as discussed further 
below in conjunction with FIG. 4, for each user in the community that indicates the availability 
of that user for receiving communication. For each user, the presence entry indicates whether the 
user is present and on what device. The presence status may indicate, e.g., whether the user is 
present, busy, away or gone (where "away" indicates that the user is around the physical 
location, but has stepped out briefly and "gone" indicates that the system has no knowledge of 
the presence of the user). The device address tab indicates the address of each device that is 
available. The presence server updates the presence and device address entries based either on 
automatic detection of presence of the user or by a process of manual registration by the user. If 
so, the presence server 300 is able to determine the address at which the user is available and the 
capabilities of the device at that address. For example, the presence server 300 can use 
information gathered from user log-ins, either to a machine or an application (or both) to 
determine presence information. In addition, determinable user activity, such as telephone, 
keyboard or mouse activity, provides presence information. In an enterprise setting, a private 
branch exchange (PBX) switch can be monitored for a user's telephone usage and to initiate 
phone calls. A user's cellular telephone can be monitored to provide data on where the user is 
currently physically located. 

The datastore accessing module 360 provides a generic interface to such different 
back-end datastores. An object registration and management module 370 is used to create and 
manage objects (users, groups, devices) and their information. Each user, group and device is 
represented in the system by objects. An access control module 350 ensures that an invoking 
object is authorized to access requested information before any information is accessed about 
any object. 

FIG. 4 is a sample table from an exemplary presence database 400 maintained by 
the presence server 300. As indicated above, the presence database 400 maintains information 
for each user in the community, including the availability of each user to receive instant 
messages. As shown in FIG. 4, the presence database 400 includes a plurality of records, such as 
record 410, each associated with a different user. For each user, identified, for example, by 
name in field 430, the presence database 400 indicates the user's presence in field 440, 
corresponding device address and capabilities in fields 450 and 460, respectively, and the user's 
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voice mailbox in field 470. The presence entry in field 440 indicates whether the user is present 
at a given device registered for the user. The device address in field 450 indicates the address of 
each device that is available for receiving instant messages for the user. The address can be any 
location or connection means, such as a phone number or URL, for example. The device 
5 capability in field 460 indicates the capability of the device, such as whether the device is text or 
voice or video capable (or some combination of the foregoing), including email and fax capable 
devices. Finally, the voice mailbox in field 470 indicates the address of the voice mailbox for 
the user. 

As previously indicated, the presence server 300 updates the presence and device 

10 address entries based on the automatic detection of the presence of the user or by a process of 
manual registration by the user, in any known manner. Thus, the presence server 300 is always 
able to determine whether a user is present. If present, the server 300 is able to determine the 
address at which the user is available and the capabilities of the device at the address. In 
addition, the manual registration process allows a user to prioritize the indicated device and 

15 presence information, thereby allowing instant messages to be delivered in accordance with the 
user's preferences. It is noted that the presence database 400 can include a SIP registry database 
FIG. 5 illustrates an exemplary "login" process 500 that can be used by users to 
affirmatively register with the presence server 300. The "login" process 500 enables a user such 
as user 1 or user 2 to declare to the presence server 300 that he or she is present by sending an 

20 appropriate message to the presence server 300. 

FIG. 6 is a flow chart describing an exemplary implementation of the client-side 
presence enabled email application process 600. Generally, the presence enabled email 
application process 600 queries for the presence of the sender of an email message. In this 
manner, the presence enabled email application process 600 can be said to make an email client 

25 incorporating the presence enabled email application process 600 a presence fetcher, as defined 
by IETF RFC 2278. 

As shown in FIG. 6, user 2 initially sends an email message to one or more 
indicated recipients (including user 1) during step 610. Thereafter, the email client of user 1 
fetches the email from user l's mailbox on the email server 130 during step 620. The email 
30 client also queries the presence server 300 to determine the presence of the sender (user 2) of the 



-11- 



502063-A-01-US (Khakoo) 



retrieved email message during step 630. The presence server 300 responds with the presence 
information during step 640. Finally, the retrieved email message is presented to the recipient 
(user 1), for example, in an "in-box," with an indication of the presence of the sender during step 
650. 

5 FIG. 7 illustrates an exemplary user interface 700 used by an email client 

incorporating features of the present invention. The email client may be embodied, for example, 
as Microsoft Outlook, as modified herein to provide the features and functions of the present 
invention. As shown in FIG. 7, the exemplary user interface 700 includes a number of 
conventional fields for presenting email messages to a user, such as subject, sender and date. In 

10 addition, the user interface 700 in accordance with the present invention includes two additional 
buttons that support the features of the present invention. A first button 710 allows the user to 
register with the presence server 300 in accordance with the "login" process 500 of FIG. 5. A 
second button 720 allows the user to check the presence of an email sender, in accordance with 
the presence enabled email application process 600 of FIG. 6. In this manner, presence 

15 information is made available on the email client and this information can be fetched for the 
originator of the message by a click of the check presence button 720. In a further variation, 
presence information can automatically be presented to the user, e.g., for each unread email 
message appearing in the list of emails presented in the user interface 700. 

The check presence button 720 can launch a dialog box that allows the user to 

20 automatically initiate a communication with the email sender at a device where the sender is 
present, such as an instant message or a "click-to-dial" telephone call. 

It is to be understood that the embodiments and variations shown and described 
herein are merely illustrative of the principles of this invention and that various modifications 
may be implemented by those skilled in the art without departing from the scope and spirit of the 

25 invention. 
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